Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[gui-tests][full-ci] retry failed scenario #11920

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

saw-jan
Copy link
Member

@saw-jan saw-jan commented Oct 2, 2024

Implements retry failing test scenario.

Condition to trigger retry:

  • full-ci in the PR titile
  • tag and nightly builds

Note

--retry and --abortOnFail options clash with each other.
So, we enable --abortOnFail on normal PR
and enable --retry if above conditions are met

What happens when retry is triggered?

  • failed test scenario is retried once
  • on retry run video recording is enabled

Other changes:

  • renamed env SCREEN_RECORD_ON_FAILURE to RECORD_VIDEO_ON_FAILURE
  • renamed config screenRecordOnFailure to recordVideoOnFailure
  • added config videoRecordLimit to limit video recordings of failed scenarios - default: 5 (because of large size)
  • moved reports related functions to separate ReportHelper file

@saw-jan saw-jan added the QA:team label Oct 2, 2024
@saw-jan saw-jan self-assigned this Oct 2, 2024
@saw-jan saw-jan changed the title [gui-tests] retry failed scenario [gui-tests][debug-gui-test] retry failed scenario Oct 2, 2024
@saw-jan saw-jan force-pushed the tests/retry-failed-test branch 3 times, most recently from d6df6e6 to cc4ed54 Compare October 3, 2024 11:37
@saw-jan saw-jan changed the title [gui-tests][debug-gui-test] retry failed scenario [gui-tests][full-ci] retry failed scenario Oct 3, 2024
@saw-jan saw-jan force-pushed the tests/retry-failed-test branch 2 times, most recently from a93e55d to 7161ffa Compare October 3, 2024 11:52
@saw-jan saw-jan marked this pull request as ready for review October 3, 2024 12:03
@@ -73,12 +73,14 @@ def get_default_home_dir():
'clientConfigDir': 'CLIENT_CONFIG_DIR',
'guiTestReportDir': 'GUI_TEST_REPORT_DIR',
'ocis': 'OCIS',
'screenRecordOnFailure': 'SCREEN_RECORD_ON_FAILURE',
'recordVideoOnFailure': 'RECORD_VIDEO_ON_FAILURE',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about writing these attributes in snake_case?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would be great. but not in this PR.

}

DEFAULT_PATH_CONFIG = {
'custom_lib': os.path.abspath('../shared/scripts/custom_lib'),
'home_dir': get_default_home_dir(),
# allow to record first 5 videos
'videoRecordLimit': 5,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
'videoRecordLimit': 5,
'video_record_limit': 5,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants